Techniques for Operational Data Warehousing

Traditionally, data warehouses have been used to analyze historical data. Recently, there has been a growing trend to use data warehouses to support real-time decision-making about an enterprise's day-to-day operations. The needs for improved query and update performance are two challenges that arise from this new application of a data warehouse. To address these needs, new data warehouse functionality is needed including: (1) better access to early query results while queries are running and (2) making the information stored in a data warehouse as fresh as possible. For the first problem, we introduce a non-blocking parallel hash ripple join algorithm to support interactive queries in a parallel DBMS. Compared to previous work, our parallel hash ripple join algorithm (1) combines parallelism with sampling to speed convergence, and (2) maintains good performance in the presence of memory overflow. We demonstrate the performance of our approach with a prototype implementation in a parallel DBMS. For the second problem, we propose two techniques to improve the efficiency of immediate materialized view maintenance. We identify two challenges for immediate materialized view maintenance: (1) In parallel RDBMSs, simple single-node updates to base relations can give rise to expensive allnode operations for materialized view maintenance. (2) Immediate materialized view maintenance with transactional consistency, if enforced by generic concurrency control mechanisms, can result in low levels of concurrency and high rates of deadlock. To address the first challenge, we present a comparison of three materialized join view maintenance methods in a parallel RDBMS, which we refer to as the naive, auxiliary relation, and global index methods. The last two methods improve performance at the cost of using more space. To address the second challenge, we extend previous high concurrency locking techniques to apply to materialized view maintenance, and show how this extension can be implemented even in the presence of indices on the materialized view.